Instrument insertion compensation

ABSTRACT

Disclosed herein are systems and techniques for compensating for insertion of an instrument into a working channel of another instrument in a surgical system. According to one embodiment, a method of compensation includes: detecting insertion of an insertable instrument into a working channel of a flexible instrument; detecting, based on a data signal from at least one sensor, a position change of a distal portion of the flexible instrument from an initial position: generating a control signal based on the detected position change; and adjusting a tensioning of a pull wire based on the control signal to return the distal portion to the initial position.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.16/018,644, filed Jun. 26, 2018, which application claims the benefit ofU.S. Provisional Application No. 62/526,008, filed Jun. 28, 2017, eachof which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to medical devices, and moreparticularly to robotically assisted surgery.

BACKGROUND

Medical procedures such as endoscopy (e.g., bronchoscopy) may involveaccessing and visualizing the inside of a patient's lumen (e.g.,airways) for diagnostic and/or therapeutic purposes. During a procedurea flexible tubular tool such as, for example, an endoscope, may beinserted into the patient's body and an instrument can be passed throughthe endoscope to a tissue site identified for diagnosis and/ortreatment. For example, the endoscope can have an interior lumen (e.g.,“working channel”) providing a pathway to the tissue site, whereinvarious tools/instruments can be inserted through the interior lumen tothe tissue site. A robotic system may be used to control the insertionand/or manipulation of the endoscope and/or the tools/instruments duringthe procedure, and may comprise at least one robotic arm that includes amanipulator assembly configured to control the positioning of theendoscope and/or tools/instrument during the procedure.

SUMMARY

The systems, techniques and devices of this disclosure each have severalinnovative aspects, no single one of which is solely responsible for thedesirable attributes disclosed herein.

Medical procedures may involve the manipulation of a flexible instrumentpositioned remotely from an operator. For example, imaging, biopsysampling, delivery of therapeutics and/or surgery can be performedwithin a lumen or luminal network to a target position within thepatient corresponding to a desired tissue site and inserting anotherinstrument through a working channel of the flexible instrument to gainaccess to the desired tissue site.

One challenge associated with existing flexible instruments for surgicalpurposes is that advancing or extending an insertable instrument throughthe working channel of the flexible instrument can cause deflection ofthe flexible instrument such that its distal end is moved from a targetposition. As the result of such deflection, the distal end of theflexible instrument can be misaligned with the tissue site.

Accordingly, certain aspects of this disclosure relate to systems andtechniques that facilitate preventing, minimizing, and/or compensatingfor deflection of the flexible instrument when another instrument isinserted through the working channel of the flexible instrument. Anotheraspect of this disclosure relates to relates to systems and techniquesthat facilitate preventing, minimizing, and/or compensating fordeflection of such a flexible instrument regardless of the source of thedeflection.

Accordingly, a first aspect of the disclosure relates to a roboticsystem. The robotic system includes a first instrument, and the firstinstrument includes a shaft with a proximal portion and a distalportion. The distal portion includes an articulable region and a distalend. The shaft includes a working channel extending therethrough. Therobotic system also includes at least one pull wire and at least onesensor configured to detect a position of the distal end of the shaft.The robotic system also includes at least one computer-readable memoryhaving stored thereon executable instructions and one or more processorsin communication with the at least one computer-readable memory. The oneor more processors are configured to execute the instructions. Theinstructions cause the system to detect, based on a data signal from theat least one sensor, a position change of the distal end of the shaft inresponse to insertion of a second instrument into the working channel ofthe shaft. The instructions further cause the system to generate atleast one control signal based on the detected position change. Therobotic system also includes a drive mechanism connected to the at leastone pull wire at the proximal portion of the shaft. The drive mechanismis configured to adjust a tensioning of the at least one pull wire basedon the at least one control signal. The adjusted tensioning facilitatesreturning the distal end of the shaft towards an initial position beforethe position change occurred.

The robotic system according to one embodiment may include one or moreof the following features, in any combination: the drive mechanism isconnected to an end effector of a robotic arm, the robotic arm and thedrive mechanism are configured to navigate the distal portion of theshaft through a luminal network of a patient to a treatment site; anelectromagnetic (EM) field generator, the at least one sensor includes afirst set of one or more EM sensors at the distal end of the shaft, andthe one or more processors are configured to execute the instructions tocause the system to calculate a first position of the first set of EMsensors within the EM field based on data from the first set of EMsensors and detect the position change of the distal end of the shaftbased on the calculated first position; the second instrument furtherincludes a second set of one or more EM sensors at the distal end, andthe one or more processors are configured to execute the instructions tocause the system to calculate a second position of the second set of EMsensors within the EM field based on data from the second set of EMsensors and generate the at least one control signal further based onthe calculated second position; the at least one sensor includes a setof one or more inertial sensors at the distal end of the shaft, and theone or more processors are configured to execute the instructions tocause the system to calculate a first position of the set of one or moreinertial sensors based on data from the set of one or more inertialsensors, and generate the at least one control signal further based onthe calculated first position; the at least one sensor includes a set ofone or more strain gauges, and the one or more processors are configuredto execute the instructions to cause the system to calculate a firstposition of the distal end of the shaft based on data from the set ofone or more strain gauges, and generate the at least one control signalfurther based on the calculated first position; the drive mechanismincludes the set of one or more strain gauges; the first instrumentincludes a leader, and the at least one sensor includes a set of one ormore cameras at the distal end of the leader; the instructions of the atleast one control signal includes commands for the drive mechanism toincrease the tension in one or more of the pull wires until the distalend of the shaft is returned to the initial position as measured by adata signal from the at least one sensor; the one or more processors area part of a workstation that includes a user interface for controllingthe system; at least one respiration sensor and the one or moreprocessors are further configured to execute the instructions to causethe system to determine, based on data from the at least one respirationsensor, a respiration pattern of a patient during acquisition of thedata signal from the at least one sensor, and distinguish the positionchange of the distal end of the shaft caused by the insertion of thesecond instrument into the working channel from a position change of thedistal end of the shaft caused by the respiration pattern of thepatient; the one or more processors are configured to execute theinstructions to cause the system to detect an identifier on the secondinstrument, and generate the at least one control signal further basedon the detected identifier; and/or the one or more processors areconfigured to execute the instructions to cause the system to detect theidentifier based on reading a radio frequency identification tag of thesecond instrument.

Embodiments discussed herein may relate to robotic systems that includea first instrument. The first instrument includes a shaft that includesa proximal portion and a distal portion. The distal portion includes anarticulable region. The shaft includes a working channel extendingtherethrough. The robotic system includes at least one pull wire. Therobotic system includes at least one sensor configured to detect, inresponse to insertion of a second instrument into the working channel, aposition of a distal end of the second instrument within the workingchannel. The robotic system includes at least one computer-readablememory having stored thereon executable instructions. The robotic systemincludes one or more processors in communication with the at least onecomputer-readable memory and configured to execute the instructions. Theinstructions cause the system to calculate, based on a data signal fromthe at least one sensor, the position of the distal end of the secondinstrument within the working channel. The instructions further causethe system to generate at least one control signal based on thecalculated position. The robotic system includes a drive mechanismconnected to the at least one pull wire at the proximal portion of theshaft. The drive mechanism may be configured to adjust a tensioning ofthe at least one pull wire based on the at least one control signal, andthe adjusted tensioning facilitates maintaining a position of the distalportion of the shaft.

Embodiments discussed herein may include one or more of the followingfeatures, in any combination: the drive mechanism is configured toadjust the tensioning of the at least one pull wire as the distal end ofthe second instrument advances to a determinable position in relation tothe articulable region; the drive mechanism is configured to adjust thetensioning of the at least one pull wire before the distal end of thesecond instrument advances to the determinable position; the drivemechanism is configured to adjust the tensioning of the at least onepull wire after the distal end of the second instrument advances to thedeterminable position; the one or more processors are configured toexecute the instructions to cause the system to detect an identifier onthe second instrument; and generate the at least one control signalfurther based on the detected identifier; the one or more processors areconfigured to execute the instructions to cause the system to determineat least one physical property of the second instrument based on thedetected identifier, the at least one physical property of the secondinstrument includes a flexural rigidity value, and the one or moreprocessors are configured to execute the instructions to cause thesystem to generate the at least one control signal further based on theflexural rigidity value; the one or more processors are configured toexecute the instructions to cause the system to determine anarticulation angle of an articulable region of the shaft, and the one ormore processors are configured to execute the instructions to cause thesystem to generate the at least one control signal further based on thearticulation angle; the one or more processors are configured to executethe instructions to cause the system to detect the identifier based onreading a radio-frequency identification (RFID) tag of the secondinstrument; and/or an EM field generator, the at least one sensorincludes a set of one or more EM sensors at the distal end of the secondinstrument, and the one or more processors are configured to execute theinstructions to cause the system to calculate a position of the set ofEM sensors within the EM field based on data from the set of EM sensorsand calculate the position of the distal end of the second instrumentwithin the working channel further based on the calculated position.

Portions of this disclosure may discuss embodiments of methods forcontrolling at least one pull wire of a first instrument. This methodincludes determining an initial position of the first instrument. Thefirst instrument includes a shaft that includes proximal and distalportions. The first instrument also includes the distal portion thatincludes an articulable region and a distal end. The first instrumentalso includes the shaft with a working channel extending therethrough.The first instrument also includes at least one pull wire. The methodalso includes detecting, based on a data signal from at least onesensor, a position change of the distal end of the shaft in response toinsertion of a second instrument into the working channel of the firstinstrument. The method also includes generating at least one controlsignal based on the detected position change of the distal end of theshaft. The method also includes adjusting a tensioning of the at leastone pull wire based on the at least one control signal and the adjustedtensioning facilitates returning the distal end of the shaft to theinitial position.

Robotic systems for controlling at least one pull wire may include oneor more of the following features, in any combination: the at least onesensor includes a first set of one or more EM sensors at the distal endof the shaft, and the detecting of the position change of the distal endof the shaft is further based on receiving data from the first set ofone or more EM sensors; the at least one sensor includes a set of one ormore inertial sensors at the distal end of the shaft, the detecting ofthe position change of the distal end of the shaft is based on data fromthe set of one or more inertial sensors; the at least one sensorincludes a set of one or more one or more strain gauges, and thedetecting of the position change of the distal end of the shaft is basedon data from the set of one or more strain gauges; the at least onesensor includes a set of one or more cameras at the distal end of thefirst instrument, the detecting of the position change of the distal endof the shaft is based on data from the set of one or more cameras;and/or determining, based on data from at least one respiration sensor,a respiration pattern of a patient during acquisition of the data signalfrom the at least one sensor, and distinguishing the position change ofthe distal end of the shaft caused by the insertion of the secondinstrument into the working channel from a position change of the distalend of the shaft caused by the respiration pattern of the patient.

Portions of this disclosure may discuss embodiments of methods forcontrolling at least one pull wire of a first instrument. Such methodsmay include detecting insertion of a second instrument into a workingchannel of the first instrument. The second instrument includes proximaland distal ends. The first instrument includes a shaft having proximaland distal portions with the distal portion having an articulableregion. The first instrument also includes at least one pull wire. Themethod also includes calculating a position of the distal end of thesecond instrument within the articulable region. The method alsoincludes generating at least one control signal based on the calculatedposition. The method also includes adjusting a tensioning of the atleast one pull wire based on the at least one control signal, whereinthe adjusted tensioning facilitates maintaining a position of the distalportion of the shaft.

The robotic system implementing methods for controlling at least onepull wire may include one or more of the following features, in anycombination: adjusting the tensioning of the at least one pull wire asthe distal end of the second instrument advances to a determinableposition in relation to the articulable region; adjusting the tensioningof the at least one pull wire before the distal end of the secondinstrument advances to the determinable position; adjusting thetensioning of the at least one pull wire after the distal end of thesecond instrument advances to the determinable position; detecting anidentifier on the second instrument, and generating the at least onecontrol signal further based on the detected identifier; determining atleast one physical property of the second instrument based on thedetected identifier, wherein the at least one control signal isgenerated further based on the at least one physical property; the atleast one physical property includes a flexural rigidity value of thesecond instrument; the detecting of the identifier includes reading anRFID tag of the second instrument; and/or the calculated position of thedistal end of the second instrument within the articulable region isbased on data from at least one EM sensor on the distal end of the firstinstrument.

Portions of this disclosure may discuss embodiments of non-transitorycomputer readable storage media. Non-transitory computer readablestorage media may have stored thereon instructions. These instructionwhen executed, cause at least one computing device to at least, for afirst instrument includes at least one pull wire, determine an initialposition of a distal end of a first instrument. The instructions furthercause at least one computing device to detect, based on a data signalfrom at least one sensor, a position change of the distal end of thefirst instrument in response to insertion of a second instrument into aworking channel of the first instrument. The instructions further causeat least one computing device to generate at least one control signalbased on the detected position change. The instructions further cause atleast one computing device to adjust a tensioning of the at least onepull wire based on the at least one control signal, and the adjustedtensioning facilitates returning the distal end of the first instrumentto the initial position before the position change occurred.

A non-transitory computer readable storage medium consistent withembodiments discussed herein may include one or more of the followingfeatures, in any combination: the at least one sensor includes a set ofone or more EM sensors at the distal end of the first instrument, andthe instructions, when executed, cause the at least one computing deviceto detect the position change of the distal end of the first instrumentbased on data from the set of one or more EM sensors; the at least onesensor includes a set of one or more inertial sensors at the distal endof the first instrument, and the instructions, when executed, cause theat least one computing device to detect the position change of thedistal end of the first instrument based on data from the set of one ormore inertial sensors; the at least one sensor includes a set of one ormore strain gauges configured to measure tensioning of the at least onepull wire, and the instructions, when executed, cause the at least onecomputing device to detect the position change of the distal end of thefirst instrument based on data from the set of one or more straingauges; the at least one sensor includes a set of one or more cameras atthe distal end of the first instrument, and the instructions, whenexecuted, cause the at least one computing device to detect the positionchange of the distal end of the first instrument based on data from theset of one or more cameras; and/or the instructions, when executed,cause the at least one computing device to determine, based on data fromat least one respiration sensor, a respiration pattern of a patientduring acquisition of the data signal from the at least one sensor, anddistinguish the position change of the distal end of the firstinstrument caused by the insertion of the second instrument into theworking channel from a position change of the distal end of the firstinstrument caused by the respiration pattern of the patient.

Portions of this disclosure may discuss embodiments of a non-transitorycomputer readable storage medium that store instructions for adjustingtensioning of pull wires. These instructions, when executed, cause atleast one computing device to at least, for a first instrument includesat least one pull wire and an articulable region, detect insertion of asecond instrument into a working channel of the first instrument. Theinstructions further cause at least one computing device to calculate aposition of a distal end of the second instrument within the articulableregion. The instructions further cause at least one computing device togenerate at least one control signal based on the calculated position.The instructions further cause at least one computing device to adjust atensioning of the at least one pull wire based on the at least onecontrol signal, wherein the adjusted tensioning facilitates maintaininga position of the distal portion of the first instrument.

The non-transitory computer readable storage medium of the sixth aspectmay include one or more of the following features, in any combination:adjust the tensioning of the at least one pull wire as the distal end ofthe second instrument advances to a determinable position in relation tothe articulable region; adjust the tensioning of the at least one pullwire before the distal end of the second instrument advances to thedeterminable position; adjust the tensioning of the at least one pullwire after the distal end of the second instrument advances to thedeterminable position; detect an identifier on the second instrument andgenerate the at least one control signal further based on the detectedidentifier; determine at least one physical property of the secondinstrument based on the detected identifier, and the at least onecontrol signal is generated further based on the at least one physicalproperty; and/or the at least one physical property includes a flexuralrigidity value of the second instrument.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an embodiment of a robotic system.

FIG. 2A illustrates a distal portion of the robotic system of FIG. 1 .

FIG. 2B illustrates deflection of the distal portion shown in FIG. 2A.

FIG. 3 illustrates the distal portion of robotic system within a luminalnetwork.

FIG. 4A illustrates a distal portion of an embodiment of a flexibleinstrument (e.g., a leader in a sheath-and-leader arrangement offlexible instruments).

FIG. 4B illustrates a robotic system including an embodiment of anelectromagnetic sensor system and a physiological sensor system.

FIG. 4C illustrates a distal portion of an embodiment of an insertableinstrument.

FIG. 5 illustrates an embodiment of a drive mechanism for controlling aflexible instrument.

FIG. 6A illustrates another embodiment of a robotic system.

FIG. 6B illustrates an embodiment of an instrument manipulator forcontrolling an instrument.

FIG. 7 illustrates an embodiment of a work station for use with arobotic system.

FIG. 8 is a flowchart of an example methodology of tracking andcompensating for deflection of a flexible instrument.

FIG. 9 is a flowchart of an example methodology of predicting andcompensating for deflection of a flexible instrument.

DETAILED DESCRIPTION Introduction

Medical procedures may involve the manipulation of an instrumentpositioned remotely from an operator. For example, imaging, biopsysampling, delivery of therapeutics and/or surgery can be performedwithin a lumen or luminal network (e.g., lung, intestine, etc.) of thebody by navigating a flexible instrument (e.g., trocars, catheters,endoscopes, etc.) to a target position within the patient correspondingto a desired tissue site and inserting another instrument through aworking channel of the flexible instrument to gain access to the desiredtissue site.

One example of a medical procedure performed with a flexible instrumentis a minimally invasive bronchoscopic technique for diagnosis andstaging of bronchial diseases called transbronchial needle aspiration(TBNA). A TBNA technique can involve manipulating a biopsy needlethrough the flexible instrument to take tissue samples at the tissuesite within the lumen of the patient. For example, a physician can usechest scans to identify the location of a mass to be biopsied and toguide positioning of the flexible instrument within the patient'sairways towards that mass. After a distal end of the flexible instrumentis positioned within the lung near the identified mass, the biopsyneedle can be advanced through the working channel of the flexibleinstrument to the location of the tissue. The tissue can then be piercedby extending the needle out of the working channel to puncture thetissue site with the needle. After sample acquisition, the needle can beretracted through the working channel.

One challenge associated with existing flexible instruments is thatadvancing or extending an insertable instrument through the workingchannel of the flexible instrument can cause deflection of the flexibleinstrument such that its distal end is deflected from a target position.The target position can be expressed, for example, at least in part asan articulation angle of the flexible instrument. By extending theinsertable instrument through the working channel, the insertableinstrument can cause a change in the articulation angle of the flexibleinstrument. As the result of such deflection, the distal end of theflexible instrument can be misaligned with the tissue site. Withoutdetection by the physician, such deflection can result in medicalprocedures performed at the wrong location within the body. This isespecially true where the tissue site, such as a lesion within a lung,has a small diameter. In some instances, manual correction for thedeflection can be performed by a physician manipulating the flexibleinstrument back into the target position. This process, however, istime-consuming, especially in medical procedures that require the use ofmultiple instruments or checking of multiple tissue sites and canrequire further consultation of radiation-based navigational aids toguide the repositioning (e.g., fluoroscopy, x-rays, computerized axialtomography scanning, etc.).

Thus, one aspect of this disclosure relates to systems and techniquesthat facilitate preventing, minimizing, and/or compensating fordeflection of the flexible instrument when another instrument isinserted through the working channel of the flexible instrument. Anotheraspect of this disclosure relates to relates to systems and techniquesthat facilitate preventing, minimizing, and/or compensating fordeflection of such a flexible instrument regardless of the source of thedeflection.

In some embodiments, a steerable endoscope may be used during a medicalprocedure. In one example, the endoscope may comprise at least twotelescoping flexible instruments, such as an inner leader portion(referred to herein as the “leader”) and an outer sheath portion(referred to herein as the “sheath”).

As used herein, the terms “flexible instrument,” “sheath,” “leader,” and“endoscope” can refer interchangeably to any type of flexible instrumentthat can inserted into the body of a patient for performing medicalprocedures. In some embodiments, but not all, the flexible instrumentscan include one or more cameras configured to facilitate navigationthrough an endoluminal pathway. These can include bronchoscopes,cystoscopes, endoscopes, colonoscopes, nephroscope, and other similarnavigable instruments. Thus, although the embodiments disclosed beloware present in the context of an endoscope or bronchoscope for insertioninto a patient's lung, other applications for flexible instruments arecontemplated herein. In some embodiments, the term “first instrument”can refer to the flexible instrument, endoscope, leader, or extendedworking channel thereof and the term “second instrument” can refer to aninsertable instrument (e.g., an instrument that performs imaging,location detection, biopsy collection, delivery of therapeutics orsurgery) that passes to the surgical site through the working channel ofthe first instrument.

As used herein, “distal” refers to the end of the scope or toolpositioned closest to the patient tissue site during use, and “proximal”refers to the end of the instrument positioned closest to the operator(e.g., a physician or robotic control system). Stated differently, therelative positions of components of the robotic systems are describedherein from the vantage point of the operator.

As used herein, the terms “about” or “approximately” refer to a range ofmeasurements of a length, thickness, a quantity, time period, or othermeasurable values. Such range of measurements encompasses variations of+/−10% or less, preferably +/−5% or less, more preferably +/−1% or less,and still more preferably +/−0.1% or less, of and from the specifiedvalue, in so far as such variations are appropriate in order to functionin the disclosed devices, systems, and techniques.

As used herein, “communicatively coupled” refers to any wired and/orwireless data transfer mediums, including but not limited to a wirelesswide area network (WWAN) (e.g., one or more cellular networks), awireless local area network (WLAN) (e.g., configured for one or morestandards, such as the IEEE 802.11 (Wi-Fi)), Bluetooth, data transfercables, and/or the like.

Various embodiments will be described below in conjunction with thedrawings for purposes of illustration. It should be appreciated thatother implementations of the disclosed concepts are possible, andvarious advantages can be achieved with the disclosed implementations.Headings are included herein for reference and to aid in locatingvarious sections. These headings are not intended to limit the scope ofthe concepts described with respect thereto. Such concepts may haveapplicability throughout the entire specification.

Example Robotic Systems

FIG. 1 illustrates an embodiment of a robotic system 100 configured tofacilitate performing medical procedure(s) at a distance, such as withina lumen of a patient. The system 100 may comprise flexible instruments,such as a sheath 120 and a leader 130 through which an insertableinstrument 140 can be inserted. As shown, with a sheath-and-leaderarrangement of flexible instruments, the leader 130 and the sheath 120are each coupled to a separate drive mechanism 154, 164, with each drivemechanism coupled to the distal end of a robotic arm 150, 160.

The distal end 122 of the sheath 120 may be configured for insertioninto a lumen of a patient (not shown) and the distal end 132 of theleader 130 can be inserted into a working channel 129 through the sheath120 and navigated to a target position within the lumen of the patient,the target position corresponding to a tissue site of the lumen of thepatient that is the target of the medical procedure(s) (see, e.g., FIG.3 ). A distal end 142 of the insertable instrument 140 can be configuredto be inserted through a working channel 139 of the leader 130 andadvanced to the distal end 132 and thereby access the tissue site toperform the medical procedure(s).

The sheath 120 can include the distal end 122, a proximal end 124, ashaft 126 extending between the distal end 122 and the proximal end 124,and an articulable region 128 of the shaft 126. The articulable region128 can be articulated with respect to a longitudinal axis of the shaft126 to facilitate navigation of the sheath 120 through the lumen of thepatient. The distal end 122 can be guided through the lumen of thepatient by articulating the articulable region 128 (e.g., via the use ofone more pull wires described in further detail below) to select apathway for the distal end 122 and by advancing the shaft 126 and thedistal end 122 through the lumen of the patient from the proximal end124. In this manner, the distal end 122 can be navigated through thelumen of the patient to the tissue site. As noted above, variousnavigational aids and systems can support this process including but notlimited to fluoroscopy, x-rays, and/or computerized axial tomography(CT) scanning. The articulable region 128 may be located between theproximal end 124 and the distal end 122, and is adjacent to the distalend 122 in the present example. This arrangement can facilitate thenavigation of the sheath 120 through the luminal network of the patient.

The leader 130 can include the distal end 132, a proximal end 134, ashaft 136 extending between the distal end 132 and the proximal end 134,and an articulable region 138 of the shaft 136. The articulable region138 can be articulated with respect to a longitudinal axis of the shaft136 to facilitate navigation of the leader 130 through the lumen of thepatient. The articulable region 138 may be located between the proximalend 134 and the distal end 132, and is adjacent to the distal end 132 inthe present example. This arrangement can facilitate the navigation ofthe leader 130 through the luminal network of the patient.

As noted above, the distal end 132 of the leader 130 can be insertedinto the proximal end 124 of the sheath 120 and supported, at least inpart, thereby. The distal end 132 of the leader 130 can be extended outof the distal end 122 of the sheath 120 and guided through the lumen ofthe patient, e.g., by articulating the articulable region 138 (e.g., viathe use of one more pull wires described in further detail below) toselect a pathway for the distal end 132 and by advancing the leader 130through the shaft 126 of the sheath 120. The sheath 120 can provide abase from which the leader 130 can be advanced and articulated to selectthe pathway through the lumen of the patient. The sheath 120 also canprovide support and facilitate steering of the leader 130. Such anadvancement technique can be used to advance the distal end 132 of theleader 130 through a luminal network of the patient to, e.g., reach atarget position adjacent a tissue site. The advancement technique can bereversed to retract the leader 130 and the sheath 120 from the luminalnetwork of the patient. In this manner, the distal end 132 of the leader130 can be navigated through the lumen of the patient to/from the tissuesite. As noted above, various navigational aids and systems can supportthis process including but not limited to fluoroscopy, x-rays, and/or CTscanning.

As shown in the example of FIG. 1 , the proximal end 124 of the sheath120 can be supported by a first robotic arm 150 configured to guide ornavigate the sheath 120 through the lumen of the patient. The firstrobotic arm 150 can include a base 152 and multiple arm segments coupledat joints extending from the base, which gives the first robotic arm 150multiple degrees of freedom. For example, one implementation of thefirst robotic arm 150 can have seven degrees of freedom corresponding toseven arm segments. In some embodiments, the first robotic arm 150includes joints that use a combination of brakes and counter-balances tomaintain a position of the first robotic arm 150. The counter-balancesmay include gas springs or coil springs. The brakes, e.g., fail safebrakes, may include mechanical and/or electrical components. Further,the first robotic arm 150 may be a gravity-assisted passive support typerobotic arm.

An end effector may comprise a drive mechanism 154 coupled to the firstrobotic arm 150 and configured to control the sheath 120. The drivemechanism 154 can include connectors to transfer pneumatic pressure,electrical power, electrical signals, and/or optical signals from thefirst robotic arm 150 to the sheath 120. The drive mechanism 154 can beconfigured to manipulate the positioning of the sheath 120 usingtechniques including direct drive, harmonic drive, geared drives, beltsand pulleys, magnetic drives, and/or the like. As described furtherbelow with reference to FIG. 5 , the drive mechanism 154 can also beconfigured to manipulate the tensioning of pull wires to articulate thearticulable region 128.

The base 152 of the first robotic arm 150 can include a source of power,pneumatic pressure, and control and sensor electronics—includingcomponents such as, for example, a central processing unit 156, databus, control circuitry, and memory 158—and related actuators such asmotors to move the first robotic arm 150. In some embodiments, the base152 includes wheels to transport the robotic system 100 and wheellocks/brakes for the wheels. Mobility of the surgical robotic system 100helps accommodate space constraints in a surgical operating room as wellas facilitate appropriate positioning and movement of surgicalequipment. Further, the mobility allows the first robotic arm 150 to beconfigured such that the first robotic arm 150 does not interfere withthe patient, physician, anesthesiologist, or other equipment. During amedical procedure, a user may control the robotic arm 150 using controldevices, for example, a command center (described in further detailbelow with reference to FIG. 7 ).

A proximal portion of the leader 130 (including a proximal end 134) canbe supported by a second robotic arm 160 configured to guide the leader130 through the working channel 129 of the sheath 120 and into/throughthe lumen of the patient. As with the first robotic arm 150, the secondrobotic arm 160 can include a base 162, multiple arm segments coupled atjoints, brakes and/or counter-balances to maintain a position of thesecond robotic arm 160. As with the base 152 of the first robotic arm150, the base 162 of the second robotic arm 160 can include a source ofpower, pneumatic pressure, and control and sensor electronics—includingcomponents such as, for example, a central processing unit 166, databus, control circuitry, and memory 168—and related actuators such asmotors to move the second robotic arm 160. In some embodiments, a base162 of the second robotic arm includes wheels and locks/brakes for thewheels. In some embodiments of the robotic system 100, the first andsecond robotic arms 150, 160 can be mounted on the same base or mountedto the patient operating table.

An end effector or drive mechanism 164 (that may be similar to drivemechanism 154) can be coupled to the second robotic arm 160 andconfigured to control the leader 130. The drive mechanism 164 caninclude connectors to transfer pneumatic pressure, electrical power,electrical signals, and/or optical signals from the second robotic arm160 to the leader 130. The drive mechanism 164 can be configured tomanipulate the positioning of the leader 130 using techniques includingdirect drive, harmonic drive, geared drives, belts and pulleys, magneticdrives, and/or the like. As described further below with reference toFIG. 5 , the drive mechanism 164 can also be configured to manipulatethe tensioning of pull wires to articulate the articulable region 138.

The distal end 142 of the insertable instrument 140 can be configured tobe inserted manually into the working channel 139 at the proximal end134 of the leader 130. For example, a handle 145 on the distal end 144of the insertable instrument 140 can be gripped by a user (e.g., aphysician) and guided down the working channel 139 to the operatinglocation. The handle 145 can include actuating mechanism(s) foroperating the insertable instrument 140 to perform the desired medicalprocedure, such as a plunging or retraction motion for acquiring samplesor therapeutics, as well as articulation for aiming or any othersuitable motion. The distal end 142 of the insertable instrument 140 canbe passed along the shafts 126, 136 and through the articulable regions128, 138 to the distal end 132 of the leader 130. The passage of thedistal end 142 of the insertable instrument 140 into the articulableregion 138 of the leader 130 can cause an undesired deflection of thearticulable region 138, as explained below with reference to FIG. 2B.Also, the passage of the distal end 142 into the articulable region 128of the sheath 120 can cause an undesired deflection of the articulableregion 128 of the sheath 120, similar to the deflection of thearticulable region 138.

As shown in the example of FIG. 2A, the articulable region 138 of shaft136 is shown without being covered by an outer casing 135 for purposesof illustration. The outer casing 135 of the leader 130 can comprise aflexible polymer material (e.g., polyurethane or polyester elastomer,etc.) and provide protection against the entry of bodily fluids into theleader 130 and ensure a smooth interface with the lumen of the patientalong the shaft 136. Furthermore, the outer casing 135 can be placedover a coiled metal band 137 that provides an outer structure to theshaft 136. Within the articulable region 138, the coiled metal band 137can be structured such that the articulable region 138 is more flexiblethan the rest of the shaft 136, such as, for example, based on thespacing of the coils in the coiled metal band 137.

FIG. 2B illustrates an example of a distal portion of the robotic system100 shown in FIG. 2A. As illustrated in broken lines, the distal end 132is navigated to a target position 118 within the lumen of the patient,the target position 118 corresponding to a desired position of a distalportion (including the distal end 132) of the leader 130 within adefined distance of, and/or aligned with the tissue site that is theobject of the medical procedure, such that the distal end 142 of theinsertable instrument 140 can be extended from the distal end 132 of theleader 130 to access the tissue site. For example, the target position118 can be at least partially expressed in terms of a location of thedistal end 132 within the lumen of the patient (e.g., a position withina coordinate system), a navigational model of the lumen of the patient,the roll, pitch, and/or yaw of the distal end 132, and/or anarticulation angle 116 of the articulable region 138.

As described above, the insertable instrument 140 can be advancedthrough the working channel 139 of the leader 130 to access the tissuesite. Upon insertion of the distal end 142 of the insertable instrument140 into the articulable region 138, the distal end 132 can be deflectedor moved out of the target position 118, shown in dashed line, to adeflected position 119, shown in solid line. Similar to the targetposition 118, the deflected position 119 can be indicated by a change inthe deflection angle 115 or by a new articulation angle 116 a of thearticulable region 138.

In one example, the deflected position 119 no longer corresponds withthe tissue site, such that the extension of the distal end 142 of theinsertable instrument 140 can be extended from the distal end 132 of theleader 130, but not have access to the tissue site, or be misalignedwith the tissue site. In one illustrative example, during the collectionof a tissue sample for a biopsy, the tissue site is a potentiallycancerous lesion having a diameter less than about 3 cm. Insertion of abiopsy needle through the working channel can cause movement of thedistal end 132 from the target position 118, thereby necessitatingcorrection by the operator of the robotic system. Otherwise, the biopsyneedle can miss the lesion and sample an incorrect tissue site withinthe lumen of the patient.

In another example, the deflection angle 115 can be as much as 15° ormore, depending on a flexural rigidity of leader 130 and a flexuralrigidity of the insertable instrument 140. Other factors that caninfluence the magnitude of the deflection angle 115 include thearticulation angle 116, the diameters of the insertable instrument 140,or the flexural rigidity of the articulable region 138. Therefore,certain aspects of the systems and techniques described herein relate todeflection or movement of the distal portion of the leader 130 from thetarget position 118 and/or automatically preventing, minimizing, and/orcompensating for the deflection.

In addition to deflection of the distal end 132 from the target position118 due to insertion of the distal end 142 of the insertable instrument140 into the articulable region 138, the distal end 132 can, in someembodiments, be deflected or moved out of the target position 118 byinsertion of the distal end 142 through the articulable region 128 ofthe sheath 120. For example, an angle (not illustrated) of thearticulable region 138 (similar to the deflection angle 115 of thearticulable region 138) can be deflected of moved by insertion of theinsertable instrument 140 into the working channel 129 and/or thearticulable region 128. The distal end 132 can be deflected or moved,accordingly. This deflection or movement can be in addition to thedeflection or movement from the change (if any) in the deflection angle115 of the articulable region 138, as described above. Therefore,certain aspects of the systems and techniques described herein relate todetection of deflection of the distal portion of the leader 130 from thetarget position 118 due to deflection or movement of the sheath 120and/or automatically preventing, minimizing, and/or compensating for thedeflection or movement.

FIG. 3 illustrates the distal portion of the robotic system 100 within aluminal network or lumen 303 of a patient, for example a lung, asillustrated. The distal end 132 of the leader 130 can be navigatedthrough the lumen 303 of the patient by advancing the proximal end 134,such as by the second robotic arm 160, and selecting a pathway throughthe lumen 303 of the patient with the distal end 132 by articulating thearticulable region 138 with the drive mechanism 164. The shaft 136 ofthe leader 130 can be advanced through the working channel 129 of thesheath 120, such as by the second robotic arm 160, and the distal end132 can be extended from the distal end 122 of the sheath 120. The shaft126 and the distal end 122 of the sheath can be navigated through thelumen of the patient by being advanced along the shaft 136 of the leader130. The sheath 120 can thereby provide a base from which the leader 130can be again advanced through the lumen 303 and articulated to selectthe pathway through the lumen 303. The sheath 120 can also providesupport and additional steering to the leader 130 by being articulatedby the drive mechanism 154. This advancement technique can be repeatedthrough the lumen 303 such that the distal end 132 of the leader 130reaches the target position 118 adjacent a tissue site 304. Theadvancement technique can be reversed to retract the leader 130 andsheath 120 from the lumen 303.

The distal end 142 of the insertable instrument 140 can be advancedthrough the interior lumen 139 of the leader 130 and out of the distalend 132 (manually and/or robotically). The distal end of the insertableinstrument 140 can thereby access the tissue site 304. As explainedabove with reference to FIG. 2B, the advancement of the insertableinstrument 140 can cause a deflection of the articulable region 138 ofthe leader and/or the articulable region 128 of the sheath 120.

Accordingly, certain aspects of the systems and techniques describedherein relate to detection of deflection of the distal end 132 of theleader 130 from the target position 118 and/or automatically preventing,minimizing, and/or compensating for the deflection. For example, it willbe appreciated that the automatic nature in at least some of theembodiments described herein can provide substantial time savings overmanual correction for deflection of the distal end 132 of the leader 130(or the distal end 122 of the sheath 120). These time savings canfacilitate faster recovery time for patients because of the reducedsurgery time, reduced fatigue on physicians, surgeons and staffperforming the medical procedures, less expensive medical procedures byreducing the amount of time necessary for completing the procedures andincreasing the accuracy of said procedures and reduced error rate forperformance of medical procedures because of the relative alertness ofthe physicians.

Another advantage of the present systems and techniques is improvedaccuracy for correcting a deflection. This improved accuracy can reducethe amount of time necessary for performing surgery by eliminating theneed to repeat medical procedures that have been performed in the wronglocation, lower the rate of false positives and false negatives forbiopsies taken in the wrong location, reduce the number of proceduresthat need to be repeated for having been done incorrectly or in thewrong location and overall increase positive patient outcomes.

Another advantage of the present systems and techniques is improveddetection of deflection. This improved detection rate can eliminate theneed for repeat medical procedures that have been performed at the wronglocation and thereby increase positive patient outcomes.

FIG. 4A illustrates an embodiment of a distal portion of a flexibleinstrument, such as, for example, the leader 130. The distal portion caninclude the articulable region 138, the distal end 132, and a distalopening of the working channel 139. The distal portion of the leader 130can further comprise tracking sensors for use in conjunction with one ormore tracking systems or sensor modalities for locating a position ofthe distal end 132 of the leader 130. Further details regarding suchtracking sensors and systems, in addition to the details herein, aredescribed in U.S. application Ser. No. 15/268,238 filed on Sep. 17, 2016and entitled “Navigation of Tubular Networks,” the entirety of which isincorporated herein by reference.

Tracking systems that monitor these tracking sensors can be used totrack and detect movement of the distal end 132, including movementssuch as those caused by insertion of the insertable instrument 140 intothe working channel 139 or from other unwanted movements of the distalend. For example, a tracking system can detect whether the distal end132 has been navigated by the system 100 into the target position 118,whether the distal end 132 has been deflected from the target position118, and/or the magnitude of the deflection from the target position118. Furthermore, each of the tracking systems can include or otherwisebe in communication with a controller such as, for example, the commandcenter 700 discussed below with reference to FIG. 7 . The controller caninclude a processor communicatively coupled with a computer readablemedium with instructions stored thereon for generating a control signalto the robotic system 100 for compensating for the measured or detecteddeflection of the distal end 132 from the target position 118 using thedata from any of the tracking systems described below.

With continued reference to the example of FIG. 4A, a number of possibletracking systems are now discussed. In one example tracking system, thedistal portion of the leader 130 can comprise one or more inertialsensors 460, such as an accelerometer and/or a gyroscope. The inertialsensor 460 can be configured to detect and/or measure changes inacceleration and output a data signal to a controller reflecting thesemeasurements. In one embodiment, the inertial sensor 460 is a 3-axismicroelectromechanical systems (MEMS)-based sensor chip with anaccelerometer and can be coupled near distal end 132 of the leader 130,for example, on the same printed circuit board as a camera 450, asillustrated in FIG. 4 , or on a different board. The accelerometer canmeasure a linear acceleration along the three different axes tocalculate the velocity and direction of the distal end 132. Thus,movements of the distal end 132 out of the target position 118 can bedetected and/or measured by the controller.

In one example, the inertial sensor 460 detects gravitational forces andprovides information regarding the location of the endoscopic toolrelative to the ground. If the inertial sensor 460 also measures thedirection of gravity, the inertial sensor 460 can provide datacontaining absolute information about the orientation of the distal end132 of the leader 130. In another example, if the leader 130 does notroll or bend up to ninety degrees, a two-axis accelerometer could alsobe used. In another example, a one-axis sensor can be useful if the axisof the accelerometer remains perpendicular to the direction of gravity,i.e., perpendicular to the ground. In yet another example, the inertialsensor 460 can comprise a gyroscope configured to measure the rate ofrotation of the distal end 132, which can then be used to calculate thearticulation of the leader 130.

The inertial sensor readings can be transmitted using digital or analogsignals through a communication protocol to a controller. The signal canbe transmitted through wiring to the proximal end of the catheter andfrom there to the controller for processing. Movements of the distal end132 out of the target position 118 can be detected and/or measured bythe controller.

As another example tracking system, the camera 450 can also be used as apart of an optical tracking system. The camera 450 in some embodimentsis a charge coupling device (CCD), or fiber optic cable extendingproximally to the distal end 132. Images from camera 450 can be idealfor navigating the distal end 132 of the leader 130 through anatomicalspaces such as the lumen of the patient and arriving at the targetposition 118. The distal end 132 can also comprises a light source, suchas an LED. In conjunction with the LEDs, the camera 450 can be used, forexample, to capture real-time video to assist with navigation within alumen of a patient. Internal bodily fluids, such as mucus, can causeproblems when navigating. Accordingly, the distal end 132 can alsoinclude component(s) for cleaning the camera 450, such as component(s)for irrigation and/or aspiration of the camera lens.

In addition to navigation, the camera can be used to detect deflectionof the distal end 132 and/or to measure the magnitude of suchdeflections. In the optical tracking system, an output or data signalfrom the camera 450 can be coupled with the controller whereby the datasignal can be processed to detect and/or measure deflection of thedistal end 132 out of the target position 118.

The distal portion of the leader 130 can also comprise one or moreelectromagnetic (EM) trackers or sensors 484 on the distal end 132 andthat may be used in conjunction with an EM tracking system 480 shown inFIG. 4B. The EM tracking system 480 can use the EM sensor 484 inconjunction with a generated electromagnetic field (EM field) to providereal-time indication of the position of the sensor within theelectromagnetic field. Thus, a position of the distal end 122 can betracked with an EM tracking system the distal end 132 includes one ormore EM sensors 484. Moreover, any movements or deflection out of thetarget position 118 can be detected and/or the magnitude of thedeflection measured using a data signal from the tracking system 480.

In EM-based tracking, a static EM field generator 486 generates an EMfield. The EM field generator 486 can be placed close to a patient 101to create a low intensity magnetic field. For example, as illustrated inFIG. 4B, the field generator 486 can be placed on a patient interfacelocation 112 for supporting a body of the patient 101. For example, thepatient interface location 112 can be a supporting platform for thepatient 101 and the field generator can be placed under the patient. Inanother example, the field generator can be held on a robotic arm orplaced around the sides of the patient interface location 112.

The static EM field generator 486 induces small-currents in sensor coilsin the EM sensor 484, which are correlated to the distance and anglebetween the sensor and the generator. The electrical signal can then bedigitized by an interface unit (on-chip or PCB) and sent viacables/wiring back to the system cart and then to the command center.The data can then be processed to interpret the current data andcalculate the precise location and orientation of the EM sensor 484,relative to the transmitters or field generator 486. Multiple sensorscan be used at different locations in the leader 130, for example, onthe articulable region 138, to calculate the positions of those EMsensors as well.

Thus, based on readings from an artificially-generated EM field, the EMsensor 484 can detect changes in field strength as it moves through thepatient's anatomy. A data signal from the EM sensor 484 can betransmitted down the shaft of the leader 130 to a controller 488 oralternatively, the controller or command center 700, for interpretationand analysis. Using the readings from EM sensor 484, display modules candisplay the EM sensor's relative position within a pre-generatedthree-dimensional model for review by the operator.

While a variety of sensors and tracking systems can be used fordetecting and measuring deflection of the distal portion of the roboticsystems 100 the choice of sensor(s) can be based at least in part on (i)the size of the sensor(s) within the endoscopic tool and (ii) the costof manufacturing and integration the sensor(s) into the sheath 120.

A set of physiological sensors 490 can be used to track physiologicalmovement of the patient. For example, the physiological sensors 490 cancomprise one or more inertial sensors be positioned on the body of thepatient to help estimate displacement of the chest surface duringrespiration. In another example, the physiological sensors 490 cancomprise an EM patch or EM respiratory sensors configured to be placedon the body of the patient and used to measure the inspiration andexpiration phases of the respiration cycle in conjunction with the EMtracking system 480. In another example, a number of additional EM patchsensors can be provided on the body of the patient (e.g., in the regionof the lumen of the patient) in order to track displacement caused byrespiration. In some embodiments, the data in the physiological sensors490 can include, for each EM patch sensor, time-dependent position datarepresenting the positions of the sensor in the EM field over time. Anumber of different EM patch sensors can be spaced apart on the body inorder to track the different displacements at these locations. Forexample, the periphery of the lungs may exhibit greater motion due torespiration than the central airways, and providing a number of EM patchsensors can enable more precise analysis of these motion effects.Furthermore, the distal end 132 of the leader 130 travels throughdifferent regions of the lumen 303 and thus experiences varying levelsof displacement due to patient respiration as it travels through thesedifferent regions. Data filtering techniques can correlate theapproximate position of the distal end 132 of the leader 130 with one ormore of the additional EM patch sensors, and can use identifieddisplacement magnitudes of these specific additional EM patch sensors tocorrect for noise or artifacts in the endoscope position signal due toairway movement, for example, via filtering/removal of respiratorymotion artifact component(s) of the endoscope position signal. This EMpatch sensor embodiment of the physiological sensors 490 is furtherdescribed in U.S. Provisional Application No. 62/480,257 filed on Mar.31, 2017 and entitled “Robotic System for Navigation of Luminal Networksthat Compensate for Physiological Noise,” the entirety of which isincorporated herein by reference.

In another example, the physiological sensors 490 comprise an acousticor other-type of respiratory sensor configured to be placed on the bodyof the patient in the region of the airways (e.g., lumen region 103) andused to measure the inspiration and expiration phases of the respirationcycle. In another example, the physiological sensors 490 can comprise anoptical sensor (e.g., an imaging device) can capture a stream of imagesof the patient's body and these images can be analyzed to identifyrespiration phase and/or displacement. In some implementations, thepatient 101 may be breathing with assistance from a ventilator duringthe procedure, and the ventilator (and/or a device communicativelycoupled to the ventilator) may provide data representing inspiration andexpiration phases of the respiration cycle.

Data from the physiological sensors can be used by the controller orcommand center 700 in conjunction with the data from the one or moretracking systems described above. By comparing this data from thephysiological sensors, movements of the patient can be filtered out ofthe data from the tracking systems, such that the filtered data isindicative of movement of the distal end 132 of the leader 130 fromdeflection due to instrument insertion, rather than patient movement(e.g., during inspiration and expiration phases of the respirationcycle).

FIG. 4C depicts an embodiment of the insertable instrument 140 with anEM sensor 482 on the distal end 142 thereof. In some embodiments, suchas in the robotic system 100 in which the insertable instrument 140 isinserted manually through the working channel 139 of the leader 130, theEM sensor 482 can be used in conjunction with the EM tracking system 480to track the progress of the distal end 142 of the insertable instrument140 through the leader 130 and/or within the lumen of the patient. Datafrom the EM sensor 482, such as data indicating the location of thedistal end 142, can also be used in conjunction with any of the othertracking mechanisms described herein. For example, the data from the EMsensor 482 can be used to initialize or terminate any of the trackingsystems described herein, for example, based on the location of the EMsensor within the leader 130 or its proximity to the distal end 132. Inanother example, the data from the sensor 482 can be used as a factor intiming adjustment of the articulable region 138, as described below inreference to FIGS. 5 and 9 . In another example, the data from thesensor 482 can be used to calculate the distance of the distal end 142from the articulable region 138 to know when the distal end 142 may beentering the articulable region 138. In another example, the data fromthe sensor 482 can be used to determine the trajectory of the distal end142 to know when the distal end 142 may be entering the articulableregion 138. In some embodiments, instead of or in addition to anEM-sensor 482, the insertable instrument 140 can comprise a metallicradio-opaque band that can be tracked or seen using conventionalradiation-based navigational aids (e.g., fluoroscopy, x-rays,computerized axial tomography scanning, etc.).

In some embodiments, the insertable instrument 140 can comprise anidentification tag, the tag corresponding to or containing informationabout the specific insertable instrument 140, and including informationsuch as, for example, the instrument's physical properties. In someembodiments, the robotic system 100 can automatically identify theinsertable instrument 140 based on the tag. For example, the tag can bean RFID tag, barcode, or the like. In some embodiments, the physicalproperties associated with the insertable instrument 140 can be encodedinto the identifier (e.g., RFID tag) and taken into account by therobotic system 100 to determine an expected deflection response of theleader 130 due to the insertion of the insertable instrument 140 intothe working channel 139.

FIG. 5 depicts an embodiment of a drive mechanism 500 configured tocontrol one or more pull wires 556. For example, the drive mechanism 500can correspond to one or more of the drive mechanisms 154 or 164, orother robotic systems described herein. Although described herein withreference to the leader 130, embodiments of the drive mechanism can alsobe used in conjunction with the sheath 120 or any other flexibleinstrument.

The drive mechanism 500 is configured to control one or more pull wires556 for manipulating the leader 130 from the proximal end 134. Bycontrolling the position of the distal end 132 the articulable region138 and by advancing the shaft 136 of the leader 130 through the lumenof the patient, the leader 130 can be navigated to the target position118, such as in response to physician inputs at a control center of thesystem 100. The pull wires 556 can control the articulation angle 116and direction of the articulable region 138. Once the distal end 132 ofthe leader 130 is at the target position 118, in some embodiments, thepull wires 556 can be locked in place to maintain the distal end in adesired position or orientation, for example, corresponding to thetarget position 118 described above with reference to FIG. 2B. Lockingthe pull wires may involve increasing the tension on the pull wires 556such that the force needed to move the leader 130 is increased.

The pull wires 556 can extend along a longitudinal length of the leader130. In some embodiments, the pull wires 556 are attached distallywithin the leader 130 with respect to an articulable region of theleader 130. The pull wires can be arranged around a periphery of theshaft 136 of the leader 130 such that increasing the tension of one pullwire will tend to articulate the articulable region in the direction ofthat pull wire. For example, four pull wires can be spaced evenly aroundthe shaft 136 with one pull wire in each cardinal direction.

The pull wires 556 can include both metallic and non-metallic materialssuch as, for example, stainless steel, Kevlar, tungsten, carbon fiber,and/or the like. The leader 130 may exhibit nonlinear behavior inresponse to forces applied by the pull wires. The nonlinear behavior maybe based on stiffness and compressibility of the shaft 126 of the leader130, as well as variability in slack or stiffness between different pullwires.

The drive mechanism 500 can include motors 551, each corresponding toand rotationally coupled with gear boxes 552. The pull wires 556 can becorrespondingly coupled with shafts 553 extending from the gear boxes552. The shafts 553 can be configured to apply a tensioning force on thepull wires 556 from rotation of the shafts 553 by rotation of thecorresponding motors 551. The pull wires 556 can be connected with theshafts 553 through pulleys 555 configured to secure the ends of the pullwires with the shafts and apply a tensioning force along the pull wiresthrough a rotational movement of the shafts 553. Alternatively, the pullwires 556 can be attached directly to the output shafts 553 with orwithout the pulleys 555.

As shown in the example of FIG. 5 , the pulleys 555 can belongitudinally aligned and concentric with output shafts 553 of themotors 551. The splines of the pulleys 555 can be designed such thatthey align and lock with splines on output shaft 553. In someembodiments, the splines are designed such that there is a singleorientation for the leader 130 to be aligned with drive mechanism 500.Locked into alignment, rotation of the shaft 553 and pulley 555 tensionsthe pull wires 556 within the leader 130, resulting in articulation ofthe articulable region 138 of the leader 130.

In some embodiments, the drive mechanism 500 can further comprise acontroller or be communicatively coupled with an external controller(e.g., the command center 700 of FIG. 7 described in further detailbelow) for controlling the rotation of the motors 551 and tensioning ofthe pull wires 556. In some embodiments, the drive mechanism 500includes rotational encoders coupled with the shafts 553 for measuringrotational position, speed, and or acceleration of the output shafts. Insome embodiments, the controller is onboard the drive mechanism 500,within a housing of the drive mechanism or a robotic arm on which thedrive mechanism 500 is mounted. The controller can be coupled with themotors 551 and configured with a processor for executing instructionsstored on a computer readable medium to control the tensioning of one ormore of the pull wires 556.

It is noted that the controller can include a processor thereon forexecuting instructions stored on a computer readable medium. Thecomputer readable medium can have instructions stored thereon forgenerating a control signal to the robotic system for compensating forthe measured or detected deflection of the distal end 132 from thetarget position 118 using the data from any of the tracking systemsdescribed above. For example, the instructions can cause the processorto process the data and generate a control signal to adjust thetensioning on specific pull wire(s) of the plurality of pull wires 556using either or both of the drive mechanisms 154, 164.

In some embodiments the instructions of the control signal are executedby the driver 500 before the distal end 142 of the insertable instrument140 is inserted through an articulable region 138. In such a preemptivemodel or approach, the distal end 132 may be deflected out of the targetposition 118 only to be returned to the target position 118 by adjustingthe tensioning on the plurality of pull wires 556 when the instrument140 is extended through the articulable region. In another example, theinstructions of the control signal can be executed after the distal end142 of the insertable instrument 140 is inserted through an articulableregion 138. In such a model or approach, the distal end 132 is returnedto the target position 118 after the instrument 140 is extended throughthe articulable region by adjusting the tensioning on the plurality ofpull wires 556. In yet another embodiment, the instructions of controlsignal can be executed as the distal end 142 of the instrument 140 isinserted through the articulable region 138. Thus, the distal end 132can be maintained substantially in the target position 118 duringadvancement of the insertable instrument 140 by adjusting the tensioningon the plurality of pull wires 556 in coordination with the advancementof the distal end 142. Any of these techniques can be performed inconjunction with data indicating the location of the distal end 142 ofthe insertable instrument 140, such as data from the EM sensor 182.

In one embodiment, the control signal can include instructions for thetension in one or more of the pull wires 556 to be gradually increasedby the drive mechanism 500 until the distal end 132 is returned to thetarget position 118. For example, the tension can be increased until thearrival at the target position 118 as measured or tracked by the opticaltracking system using the camera 450 to determine the position of thedistal end 132. As another example, the tension can be increased untilthe arrival at the target position 118 as measured or tracked by the EMtracking system 480 or the inertial tracking system to determine theposition of the distal end 132. In some cases, the tensioning of a pullwire can axially compress a flexible instrument, thereby shortening thedistal length of the flexible instrument. In such cases, the controlsignal may compensate for this shortening by causing the flexibleinstrument to be inserted in the anatomy by a distance that corrects forthe axial compression.

The drive mechanism 500 can include a tension sensing system formonitoring the movement and position of the distal portion of the leader130. This tension sensing system can be configured to detect and/ormeasure a deflection or movement of a distal end 132 of the leader 130by detecting change in the tensioning of the pull wires 556 caused bysuch deflection. For example, the drive mechanism 500 can monitorspecific pull wires of the pull wires 556 to monitor these specific pullwires for an increase or decrease in tension.

For example, the drive mechanism 500 can comprises one or moreelectrical strain gauges 554 for detecting/measuring the deflection ofthe pull wire(s) 556 based on any measured changes in the tensioning ofthe pull wire(s) 556. For example, in certain embodiments, the straingauges 554 are coupled between motor mounts 558 corresponding to each ofthe motors 551 and strain gauge mounts 557. Strain gauges 554 can bepotted and soldered to the strain gauge mount 557 and attached usingscrews to motor mounts 558 respectively. The strain gauges 554 can beheld in place to their respective motor mount using side screws. Thegauge wiring in the strain gauges 554 can be vertically arranged todetect any vertical strain or flex in the drive mechanism which ismeasured as horizontal displacement by the motor mount 558 relative tothe strain gauge mount 557. The amount of strain can be measured as aratio of the horizontal displacement of the tip of strain gauge 554 tothe overall horizontal width of the strain gauge 554. Accordingly, thestrain gauge 554 can ultimately measure the force exerted on the shaft553 by the pull wire 556.

The strain gauges 554 can be configured such that any change in thetensioning of any of the pull wires 556 can be detected and measured.The drive mechanism 500 can be calibrated such that the strain measuredin the strain gauges 554 can be correlated to a position of the leader130, such as the position of the distal end 132 and/or the deflectionangle 116 of the articulable region 128. Any change in the position ofthe distal end 132 can thus be detected and/or measured.

A data signal from the strain gauge 554 and/or from circuitry coupledwith the strain gauge 554 can be delivered to a controller (e.g., withinthe drive mechanism 500 shown in FIG. 5 or the command center 700 inFIG. 7 ). This data signal can contain data indicating the changes inthe tensioning of the pull wire(s) 556, the movement of the pull wire(s)556 and/or the movement of the sheath 120. Accordingly, deflection ofthe leader 130, such as by the insertion of an insertable instrument 140within a working channel 139 or through an articulable region 138 of theleader 130, can be detected and/or measured by drive mechanism 500 orcomponent(s) thereof.

FIG. 6A illustrates an embodiment of a robotic system 600. Similar tothe robotic system 100, the system 600 can comprise a sheath 620, aleader 630, and an insertable instrument 640. The leader 630 isconfigured to be inserted into a lumen of a patient (not shown) andnavigated within the luminal network of the patient. For example, thesheath 620 and the leader 630 can have the same or similar structure andmechanics as the sheath 120 and the leader 130 described above,respectively.

The leader 630 can include a distal end 632, a proximal end 634, a shaft636 extending between the distal end 632 and the proximal end 634, andan articulable region 638 of the shaft 636. The articulable region 638is configured to be articulated with respect to a shaft 636 tofacilitate navigation of the leader 630 through the lumen of the patientafter being extended from the distal end 622 of the sheath 620. Thedistal end 632 can be guided through the lumen of the patient byarticulating the articulable region 638 to select a pathway for thedistal end 632 and by advancing the shaft 636 and the distal end 632through the lumen of the patient from the proximal end 634. Similar tothe above, the sheath 620 can be advanced along with the leader 630 andprovide support thereto, such as for articulating the articulable region638 and further advancing the leader 630. In this manner, the distal end632 can be navigated through the lumen of the patient to a targetposition (e.g., see target position 118 in FIG. 2B). The articulableregion 638 is located between the proximal end 634 and the distal end632, and is adjacent to the distal end 632 in the present example. Thisarrangement can facilitate the navigation of the leader 630 through theluminal network of the patient Like the distal end 132 of the leader130, the leader 130 can include sensors for navigating the lumen of thepatient, such as those described in relation to FIGS. 4A-5 . Any of theabove described tracking systems can be used to track the location ofthe distal end 632 or detect changes in the location.

Similar to the sheath 120 shown in FIG. 1A, the sheath 620 shown in FIG.6A can include a distal end 622, a proximal end 624, a shaft 626extending between the distal end 622 and the proximal end 624, and anarticulable region 628 of the shaft 626. The articulable region 628 canbe articulated with respect to a shaft 626 to facilitate navigation ofthe sheath 620 through the lumen of the patient and to provide supportto the leader 630.

A proximal portion including the proximal end 624 of the sheath 620 canbe supported by a first robotic arm 650 configured to guide or navigatethe sheath 620 through the lumen of the patient and coupled with a drivemechanism 654. The first robotic arm 650 and drive mechanism 654 caninclude structural and functional features similar to the first roboticarm 150 and drive mechanism 154 discussed above in the robotic system100. The first robotic arm 650 can include a base 652 and multiple armsegments coupled at joints extending from the base 652, a source ofpower, pneumatic pressure, and control and sensor electronics—includingcomponents such as, for example, a central processing unit 656, databus, control circuitry, and memory 658—and related actuators such asmotors to move the first robotic arm 650. The base 652 can includewheels to transport the robotic system 600 and wheel locks/brakes forthe wheels. As described further above with respect to FIG. 5 , thedrive mechanism 654 can also manipulate the tensioning of pull wires toarticulate the articulable region 628.

A proximal portion including the proximal end 634 of the leader 630 canbe supported by a second robotic arm 660 configured to guide or navigatethe leader 630 through the lumen of the shaft 626 of the sheath 620 andinto the lumen of the patient. As with the first robotic arm 650, thesecond robotic arm 660 can include a base 662, multiple arm segmentscoupled at joints, brakes and/or counter-balances to maintain a positionof the second robotic arm 660.

An end effector or a drive mechanism 664 can be coupled with the secondrobotic arm 660 to control the leader 630. Like the drive mechanisms154, 164, the drive mechanism 664 can include connectors to the leader630 and manipulate the positioning of the leader 630. As describedfurther above with respect to FIG. 5 , the drive mechanism 664 can alsomanipulate the tensioning of pull wires to articulate the articulableregion 638. The base 662 of the second robotic arm 660, similar to thebase 152 of the first robotic arm 150, can include a source of power,pneumatic pressure, and control and sensor electronics, a centralprocessing unit 666, data bus, control circuitry, and memory 668, andrelated actuators such as motors to move the second robotic arm 660.During procedures, a user may control the second robotic arm 660 usingcontrol devices, for example the command center.

Similarly, a proximal end 644 of the insertable instrument 640 can besupported by a third robotic arm 670 and/or an instrument manipulator674 and configured to guide the insertable instrument 640 and controlthe insertable instrument 640 to perform the medical procedures. Thethird robotic arm 670 and instrument manipulator 674 can includestructural and functional features similar to the first and secondrobotic arms 650, 660 and the robotic arms in the robotic system 100.Here, however, the insertable instrument 640 is inserted and guided downthe working channel 639 of the leader 630. As with the first robotic arm650, the third robotic arm 670 can include a base 672, multiple armsegments coupled at joints, brakes and/or counter-balances to maintain aposition of the third robotic arm 670. The base 672 of the third roboticarm 670 can include a source of power, pneumatic pressure, and controland sensor electronics—including components such as, for example, acentral processing unit 676, data bus, control circuitry, and memory678—and related actuators such as motors to move the third robotic arm670. The base 672 of the third robotic arm 670 can include wheels andlocks/brakes for the wheels.

Alternatively, the insertable instrument 640 can be configured tooperated manually, such as by the physician. In such an embodiment, theinsertable instrument 640 can include the EM sensor 482 configured toprovide data that can track the location of the insertable instrument640 within the working channel 629 or into the lumen of the patient, asdescribed above in relation to the EM sensor 482.

The insertable instrument 640 can have various physical characteristicssuch as a diameter small enough that it can be inserted into the workingchannel 629, length sufficient to extend through the leader 630, weight,and flexural rigidity along its length. In some embodiments, theinsertable instrument 640 comprises an identification tag, the tagcorresponding or containing information about the specific insertableinstrument 640, and including information such as the instrument'sphysical properties. In some embodiments, the robotic system 600 canautomatically identify the insertable instrument 640 based on the tag.For example, the tag can be an RFID tag, barcode, or the like. In someembodiments, the physical properties associated with the insertableinstrument 640 are taken into account by the robotic system 600 todetermine an expected deflection response of the leader 630 due to theinsertion of the insertable instrument 640 into the leader 630.

In some embodiments, to increase the reach of the system 600 into thelumen of the patient, for example, to gain access to the periphery of apatient's lung, an insertable instrument such as an extended workingchannel having a smaller diameter than the leader 630 can be insertedinto a working channel 639 of the leader 630 and extended out into thelumen of the patient at a distal end 632 of the leader 630. The distalend of the extended working channel can then be extended or navigated toa target position 618, corresponding to a tissue site of the lumen ofthe patient for implementing the medical procedure. A distal end 642 ofthe insertable instrument 640 is configured to be inserted through aworking channel of the extended working channel and advanced to thedistal end thereof and access the tissue site for performing the medicalprocedures. The extended working channel can thereby increase the accessor reach of the leader 130 alone.

In some medical procedures, the size and/or flexibility of the leader630 or sheath 620 increase the possibility of damage to the lumen of thepatients by the passage of the leader 630. Therefore, it may bedesirable in some medical procedures to use only the leader 630 withoutthe sheath 620. For example, the leader 630 can be advanced into thelumen of the patient and controlled using the second robotic arm 660. Asthe leader 630 in such an embodiment may have a larger diameter than theleader 630 used in conjunction with the sheath 620, the leader 630 maybe used with or without an extended working channel, such as to gainaccess to the periphery of a patient's lung.

FIG. 6B depicts an embodiment of an instrument manipulator configured tocontrol advancement and operation of one or more instruments. Althoughdescribed below with reference to the robotic system 600 forillustrative purposes, the instrument manipulator 674 as describedherein can in some embodiments be used in conjunction with roboticsystem 100, such as to replace the manual control of the insertableinstrument 140. With reference to FIG. 6B, the instrument manipulator674 can be configured to support the proximal end 644 of the insertableinstrument 640 and, in conjunction with the third robotic arm 670. Theinstrument manipulator 674 and/or robotic arm 670 can navigate thedistal end 142 of the insertable instrument 140 through the workingchannel 639 of the leader 630 to access the tissue site.

In one example, the insertable instrument 640 can be a needle assembly.The needle assembly includes a jacket 647, needle 645, and a tubularelongate shaft 649 connected to the needle. The third robotic arm 670can be configured to locate, and maintain positioning of, the needleassembly. The third robotic arm 670 may include a first grip portion 682for controlling and administering therapeutics and two additional gripportions 684, 886 that can secure the shaft 649 and jacket 647,respectively. In some embodiments, the first, second, and third gripportions 682, 684, 686 can be on the same robotic arm, as describedabove, or on different robotic arms in any combination. The first gripportion 682 can include one or more actuators 688 for controlling, forexample, a syringe and/or robotically controlling a plunger of thesyringe. The third grip portion 686 may maintain stationary positioningof the jacket 647. The second grip portion 684 can be configured to movethe proximal end of the shaft 649 proximally and distally to move theneedle 645 in and out of the jacket 647 and/or to effect sampling of thetissue site.

Other examples of instruments include but are not limited to forceps,brushes, scalpels, lasers, augers, cameras, and probes. In someembodiments, the insertable instrument 640 can be substituted for otherembodiments of instruments intra-operatively to perform multipletreatments aspects in a single procedure. As another example, theinstrument manipulator 674 can include a drive mechanism having at leastone pull wire, similar to drive mechanisms described herein, such as foractuating forceps using the at least one pull wires. In other examples,the instrument manipulator can include various motors, pressureregulators, electrical connections, etc. for operating the insertableinstrument 640 to perform various medical procedures. Thus, theinstrument manipulator 674 can have various configurations toaccommodate a variety of instrument types.

FIG. 7 illustrates the command center 700 that can be used, for example,in conjunction with the robotic systems described above. The commandcenter 700 includes a console base 701, display modules 702, e.g.,monitors, and control modules, e.g., a keyboard 703 and joystick 704. Insome embodiments, one or more of the command center 700 functionalitiesmay be integrated into the controller on the robotic system or anothersystem communicatively coupled to the robotic system. A user 705, e.g.,a physician, may remotely control the robotic system from an ergonomicposition using the command center 700.

The console base 701 may include a central processing unit, a memoryunit, a data bus, and associated data communication ports that areresponsible for interpreting and processing signals such as data fromany of the tracking systems described above including but not limitedto: the tension sensing system, the optical tracking system, theinertial tracking system, the EM tracking system, and the physiologicaltracking system.

The console base 701 can also process commands and instructions providedby the user 705 through the control modules 703 and 704. In addition tothe keyboard 703 and joystick 704 shown in FIG. 7 , the control modulesmay include other devices, for example, computer mice, trackpads,trackballs, control pads, system controllers such as handheld remotecontrollers, and sensors (e.g., motion sensors or cameras) that capturehand gestures and finger gestures. A system controller can include a setof user inputs (e.g., buttons, joysticks, directional pads, etc.) mappedto an operation of the instrument (e.g., articulation, driving, waterirrigation, etc.).

The user 705 can control a flexible instrument (e.g., the sheath 120,leader 130, sheath 620, or leader 630, although described herein interms of the leader 130) using the command center 700 in, for example, avelocity mode or position control mode. In velocity mode, the user 705directly controls pitch and yaw motion of a distal end 132 of the leader130 based on direct manual control using the control modules. Forexample, movement on the joystick 704 may be mapped to yaw and pitchmovement in the distal end 132 of the leader 130. The joystick 704 canprovide haptic feedback to the user 705. For example, the joystick 704may vibrate to indicate that the leader 130 cannot further translate orrotate in a certain direction. The command center 700 can also providevisual feedback (e.g., pop-up messages) and/or audio feedback (e.g.,beeping) to indicate that the leader 130 has reached maximum translationor rotation. The haptic and/or visual feedback can also be provided dueto the system operating in a safety mode during patient expiration asdescribed in more detail below.

In position control mode, the command center 700 can use athree-dimensional (3D) map of a patient lumen and input fromnavigational sensors as described herein to control a surgicalinstrument, e.g., the leader 130. The command center 600 providescontrol signals to robotic arms of the robotic system 100 to manipulatethe distal ends 122 (or distal end 632) to the target position 118, suchas by control of the articulation angle 116 of the articulable regions128.

In some embodiments, a model of the leader 130 is displayed with the 3Dmodels to help indicate a status of a surgical procedure. For example,the CT scans identify a lesion in the anatomy where a biopsy may benecessary. During operation, the display modules 702 may show areference image captured by the leader 130 corresponding to the currentlocation of the leader 130. The display modules 702 may automaticallydisplay different views of the model of the leader 130 depending on usersettings and a particular surgical procedure. For example, the displaymodules 702 show an overhead fluoroscopic view of the leader 130 duringa navigation step as the leader 130 approaches an operative region of apatient.

Example Deflection Compensation Techniques

In accordance with one or more aspects of the present disclosure, FIG. 8depicts a flowchart of an implementation of a tracking compensationprocess 800 for detecting and compensating for a deflection of thedistal end of a flexible instrument. The process 800 is described withreference to the robotic system 100 for illustrative purposes; however,the process 800 may be implemented on other suitable robotic systems.

The process 800 can begin based on conditions of and/or inputs into therobotic system 100. For example, process 800 can begin based on or inresponse to the position of a first instrument, e.g., the insertableinstrument 140 within the working channel 139 of the leader 130. Forexample, the process 800 can begin based on a specific position of thedistal end 142 of the insertable instrument 140 within the workingchannel 139, such as proximity to an articulable region 138 of theleader 130 (e.g., within about 10 cm) or the distal end 132 of theleader 132. In another example, the system 100 can determine that theuser is manually triggering the process 800 through a user interface oruser input device, such as at the command center 700. In yet anotherexample, the process 800 can be triggered automatically as the system100 recognizes that the distal end 132 of the leader 130 has beennavigated to the target position 118 using one of the above-describedtracking systems. In still another example, the process 800 is initiatedin response to there being no further user inputs or commands to move ormanipulate any of the controllable elements of the system 100.

At block 810, the system 100 can determine (e.g., detect or measure) aninitial position of a first instrument. The first instrument maycomprise: a shaft comprising proximal and distal portions, the distalportion comprising an articulable region and a distal end, the shaftcomprising a working channel extending therethrough; and at least onepull wire. Block 810 may involve determining an initial position of adistal end of a flexible instrument (e.g., distal end 132 of the leader130). In some implementations, the initial position can correspond tothe target position 118.

Any of the above-described tracking systems for monitoring the positionof the distal end 132 can be used to detect the initial position of thedistal end 132. For example, the EM tracking system 480 can relay dataabout the sensor 484 to the controller indicating the initial positionof the distal end 132; and/or the inertial tracking system can relaydata about the sensor 460 indicating the initial position of the distalend 132. The electrical strain gauges 554 can relay data based on thetensioning of the pull wire(s) 556 indicating the initial position ofthe distal end 132. The camera 450 of the optical tracking system canrelay data based on optical positioning indicating the initial positionof the distal end 132.

At block 820, the system 100 can detect, based on a data signal from atleast one sensor, a position change (e.g., deflection) of the distal endof the shaft in response to insertion of a second instrument into theworking channel of the first instrument. For example, block 820 mayinvolve detecting, based on a data signal from at least one sensor, aposition change of the distal end 132 of the leader 130 from the initialposition, e.g., in response to insertion of an insertable instrument 140into a working channel 139 of the leader 130. Any of the above-describedtracking systems for monitoring the position of the distal end 132 canbe used to detect the deflection of the distal end 132. The controllercan receive data indicating a deflection from the tension sensingsystem, the optical tracking system, the inertial tracking system,and/or the EM tracking system 480. For example, the EM tracking system480 can relay data about the sensor 484 indicating a change in theposition of the distal end 132 to the controller; the inertial trackingsystem can relay data about the sensor 460 indicating a change in theposition of the distal end 132 and/or the deflected position 119 to thecontroller; the optical tracking system can relay data from the camera450 indicating a change in the position of the distal end 132 to thecontroller; and/or the tension sensing system 500 can relay data fromthe strain sensors 554 indicating a change in the articulation angle 116of the articulable region 138 to the controller.

In some examples, measuring the position change can involve filteringout physiological movement (e.g., the respiration pattern) of thepatient that is different from and/or not indicative of the deflectionof the distal end 132 (e.g., changes in the articulation angle 116). Forexample, at, before, or after block 820 in the process 800, a datasignal from the physical physiological movement sensors 490 can bereceived by the controller. The system 100 can thus take into account(e.g., compensate for) detected position changes of the distal end 132due to physiological movement of the patient.

At block 830, the system 100 can generate at least one control signalbased on the detected position change of the distal end of the shaft.For example, block 830 may involve generating at least one controlsignal based on the data from the tracking system indicating theposition change of the distal end 132. The generated control signal canbe at least partially based on the initial position, the magnitude,direction and/or angle of any detected deflection, and/or the signalfrom the physiological movement sensors 490.

The control signal can include instructions for returning the distal end132 back to an initial position. In some embodiments, the control signalcan include instructions for returning the distal end 132 back to thetarget position 118. For example, the control signal can includeinstructions for the drive mechanism 164 to adjust a tensioning of atleast one of the pull wires 556 of the leader 130 to compensate for thedeflection and thereby return the distal end 132 back to the initialposition. In the alternative, or in addition, the control signal caninclude instructions for the drive mechanism 154 to adjust a tensioningof at least one of the pull wires of the sheath 120 to return the distalend 132 of the leader 130 back to its initial position.

A block 840, the system 100 can adjust a tensioning of the at least onepull wire based on the at least one control signal, wherein the adjustedtensioning facilitates returning the distal end of the shaft to theinitial position. For example, block 840 may involve adjusting atensioning of at least one pull wire of the leader 130 and/or sheath 120based on the at least one control signal. For example, the drivemechanism 164 and/or drive mechanism 154 can execute the instructions inthe control signal and adjust a tensioning of one or more pull wires (ofthe leader 130 and/or the sheath 120) to return the distal end 132 tothe initial position, and thereby compensate for any deflection of thedistal end 132 due to the insertion of the insertable instrument 140.

The system 100 can end the process 800 based on any of severalconditions. In one example, the system 100 ends the process 800 upondetecting that the distal end 132 of the leader 130 has returned to theinitial position after the detected position change. In another example,the system 100 ends the process 800 in response to receiving anoverriding input control signal from the user, for example, via thecommand center 700. In another example, the system 100 ends the process800 based on detecting a manual input by the user. In another example,the process 800 can be ended by the position and/or direction ofmovement (e.g., retraction) of the insertable instrument 140 within theworking channel 139 of the leader 130. For example, a movement detectedby the EM tracking system 480 can indicate retraction of the insertableinstrument 140 from the articulable region 138 and/or from the workingchannel 139.

Alternatively, having detected one deflection of the distal end 132 fromthe initial position, the process 800 can be repeated as the position ofthe distal end 132 continues to be tracked or monitored by the system100. Subsequent detections of deflection of the distal end 132,generating control signals, and adjusting of tensioning of the pullwires 556 of the leader 130 and/or sheath 120 can be continued asoutlined above.

In another implementation, the process 800 described above can beperformed using the robotic system 600 and by detecting deflection ofthe distal end 632 of the leader 630. At block 810, the system 600 candetect an initial position (e.g., target position 618) of the distal end632 of the leader 630 using any of the above-described tracking systems.At block 820, the system 600 can detect a position change (e.g.,deflection) of the distal end 632 of the leader 630, such as, forexample, by using the tension sensing system or the EM tracking system480, and/or any other tracking systems described herein. In thealternative, or in addition, any of the above-described tracking systemscan be used to detect the deflection of the distal end 622 of the sheath620—which can also be indicative of a deflection of the distal end 632of the leader 630 requiring compensation.

At block 830, the system 600 can generate at least one control signalbased on the data from the tracking system indicating the positionchange, the detected deflection, the physiological movement sensors 490,and/or the magnitude of the deflection. The control signal can includeinstructions for returning the distal end 632 back to the initialposition (e.g., instructions for the drive mechanism 664 to adjust atensioning of at least one of the pull wires 556 of the leader 630). Inthe alternative, or in addition, the control signal can includeinstructions for the drive mechanism 654 to adjust a tensioning of atleast one of the pull wires of the sheath 620 to return the distal end632 of the leader 630 back to its initial position.

A block 840, the drive mechanism 664 and/or drive mechanism 654 canexecute the instructions in the control signal and adjust a tensioningof one or more pull wires of the leader 630 and/or the sheath 620 toreturn the distal end 632 of the leader 630 to the initial position.

In accordance with one or more aspects of the present disclosure, FIG. 9depicts a flowchart of an example process for compensating fordeflection of a first instrument, e.g., a flexible instrument, based oncontrolling at least one pull wire of the first instrument. The process900 is described with reference to the robotic system 100 forillustrative purposes; however, the process 900 may be implemented onother suitable robotic systems.

The process 900 can begin based on any of several conditions or inputsto the system 100. At block 910, the system 100 can detect insertion ofa second instrument into a working channel of the first instrument,wherein the second instrument may comprise proximal and distal ends. Thefirst instrument may comprise: a shaft comprising proximal and distalportions, the distal portion comprising an articulable region; and atleast one pull wire. The condition can be based on the position of theinsertable instrument 140 (e.g., a specific position of the distal end142 of the insertable instrument 140) within the working channel 139 ofthe leader 130, such as the proximity of the insertable instrument 140to an articulable region 138 or distal end 132 of the leader 130 (e.g.,within about 2 cm, 5 cm, 10 cm, or any other suitable thresholddistance). Additionally or alternatively, the system 100 can determinethat the user is manually triggering the beginning of the process 900through a user interface, such as at the command center 700. In oneexample, the process 900 can be initiated automatically as the system100 recognizes that the distal end 132 is in the target position 118(e.g., based on the system 100 automatically detecting this conditionand/or based on the system 100 receiving a user input indicative of thiscondition). In yet another example, the process 900 is initiated inresponse to there being no further user inputs or commands to move ormanipulate any of the controllable elements of the system 100.

In one embodiment, the system 100 can track the advancement of theinsertable instrument 140 through the working channel 139. For example,the EM tracking system 480 can relay data about the sensor 482 of theinsertable instrument 140 to the controller indicating the position ofthe distal end 142 within the working channel 139. The position with theworking channel 130 can include the proximity to and/or arrival of thedistal end 142 at the articulable region 138 and/or at the distal end132 of the leader 130.

At block 920, the system 100 can calculate a position of the distal endof the second instrument within the articulable region. For example,block 920 may involve calculating the position of the distal end of thesecond instrument within the articulable region based on data from theEM tracking system 480 and/or robot control data.

At block 930, the system 100 can generate at least one control signalbased on the calculated position of the distal end of the secondinstrument within the articulable region. In other embodiments, the atleast one control signal may be based on the predicted deflection of thefirst instrument resulting from the calculated position of the distalend of the second instrument within the articulable region. The controlsignal can include instructions for preventing the distal end 132 fromdeflecting from the target position 118 or otherwise returning thedistal end 132 at the target position based on the predicted deflection.For example, the control signal can include instructions for the drivemechanism 164 to adjust a tensioning of at least one of the pull wires556 of the leader 130 to prevent or minimize any deflection that mayotherwise occur. In some embodiments, the control signal can includeinstructions for the drive mechanism 154 (or both drive mechanisms 154,164) to adjust a tensioning of at least one of the pull wires tomaintain the distal end 132 at the target position 118. It is to beappreciated that in some cases the leader may compress as a result ofthe increased tensioning on the pull wires. In such cases, the controlsignal may also instruct a robotic arm controlling the leader to causethe leader to be inserted a specified distance that is related to thecompression that will be experienced by the leader. In this way, thecombination of the insertion and the compression along the length of theleader is such that the distal end of the leader maintains its locationwithin the anatomy (e.g., the target position 118).

At block 940, the drive mechanism 500 of the system 100 can adjust atensioning of the at least one pull wire based on the at least onecontrol signal, wherein the adjusted tensioning facilitates maintaininga position of the distal portion of the shaft. For example, block 940may involve executing instructions in the control signal and adjustingthe tensioning of the pull wire(s) 556 of the leader 130. In someembodiments, the instructions contained in the control signal areexecuted in coordination with a determinable position of the distal end142 of the insertable instrument 140 within the working channel 139. Forexample, the determinable position can be calculated using the data fromthe EM sensor 482 on the insertable instrument 140. In otherimplementations of the method, such as using system 600 described above,the determinable position can be calculated based on the known positionsof the robotic arms and their relations to one another. In someimplementations, the instructions of the control signal are executedbefore the distal end 142 of the insertable instrument 140 is insertedto a particular determinable position, such as within the articulableregion 138. In such a preemptive model or approach, the distal end 132may be temporarily deflected out of the target position 118 by thecontrol signal, but the distal end 132 returns to the target position118 once the insertable instrument 140 is advanced to a seconddeterminable position, such as the articulable region 138 or the distalend 132. In another embodiment, the instructions of the control signalcan be executed after the distal end 142 of the insertable instrument140 is advanced to the distal end 132 or through the articulable region138. In such a model or approach, the distal end 132 is temporarilydeflected and then returned to the target position 118 after the controlsignal is fully executed.

In another example of the preemptive approach, system 100 minimizes theextent or magnitude of deflection of the distal end 132 from the targetposition by executing the control signal in coordination with thedeterminable position of the distal end 142 of the insertable instrument140 (e.g., substantially concurrently), thereby minimizing the amount ofdeflection experienced by the distal end 132. For example, the controlsignal can be executed to adjust the tensioning of the one or more pullwires in increments as the distal end 142 of the insertable instrumentis advanced through the articulable region 138.

The end of the process 900 can be triggered by an overriding inputcontrol signal from the user or the command center 700 or anothercomponent of the system 100. In one example, the end of the process 900can also be triggered by the system 100 receiving a manual input by theuser. In another example, the end of the process 900 can be triggered byautomatic detection of a position of the distal end 142 of theinsertable instrument 140 within the working channel 139, such as apositional indication that the insertable instrument 140 is beingretracted from the working channel or has been retracted from thearticulable regions 138.

In accordance with one or more aspects, there is a provided a processthat involves calculating a predicted deflection of the firstinstrument, e.g., the distal end 132 from target position 118 due to theinsertion of the insertable instrument 140 through the articulableregion 138. The calculated predicted deflection can be based on one ormore factors such as, for example, the location of the insertableinstrument 140 within the working channel 139, the location of theinstrument relative to an articulable region 138 of the leader 130,and/or the articulation angle 116 of the articulable region 138. Thecalculated predicted deflection can also be based on, for example, thephysical properties of the leader 130 and/or the insertable instrument140 (including pull wires thereof), such as, length, diameter, weight,elasticity, and/or flexural rigidity, etc.

In some embodiments, the system 100 calculates the predicted deflectionby recognizing the insertable instrument 140 and/or correlating it withknown physical characteristics of the insertable instrument 140. Forexample, the insertable instrument 140 can be identified based on itstag, such as an RFID tag that can include information (e.g., physicalproperties) about that particular instrument. The insertable instrument140 can be correlated to a set of physical properties of the instrumentsuch as, for example, the instrument diameter, length, weight, and/orflexural rigidity of specific portions of the insertable instrument 140.This information can be taken into consideration by the system 100 whencalculating the predicted deflection. In some embodiments, the locationof the distal end 142 of the insertable instrument 140 is taken intoaccount in calculating the compensation based on data from the EM sensor482 used in conjunction with the EM tracking system 480.

In some embodiments, the predicted deflection may be calculated by acontroller or computing device communicatively coupled with the system100 using the factors and physical properties described above and apredictive, mathematical model of the leader 130 and/or insertableinstrument 140. In other embodiments, the predicted deflection isknown/stored in memory or looked up in a database. In such anembodiment, the appropriate control signal and/or predicted deflectioncan be looked up in a corresponding database based on information aboutthe system 100 (e.g., physical properties of the leader 130 orinsertable instrument 140, articulation angle 116, tension in the one ormore pull wires 554, or other properties of the system 100). Forexample, given a known articulation angle 116 of the articulable region138, a known leader 130, and a known insertable instrument 140, thepredicted deflection can be looked up in a database correlating thesevariables.

Alternatively, having calculated one predicted deflection of the distalend 132, the above process 900 can be repeated as the position of theinsertable instrument 140 continues to be tracked or monitored by thesystem 100. Subsequent calculations predicting the deflection of thedistal end 132 can be processed as outlined above until the end of theprocess 900.

In another implementation, the process 900 described above can beperformed using the robotic system 600 and by predicting deflection ofthe distal end 632 of the leader 630. At block 910, the system 600tracks the position of the insertable instrument 640 within the workingchannel 639 based on the position, models, sensors, and/or controls ofthe system 600. For example, the system 600 can track the position ofthe insertable instrument 640 based on the robotic insertion data of therobotic arms 650, 660, and/or 670 which guide and support the sheath620, leader 630, and/or insertable instrument 640, respectively. Thisrobotic insertion data can, for example, include data indicating theposition and orientation of the distal end 642 of the insertableinstrument 640 relative to the articulable region 638 and/or distal end632 of the leader 630, for example, as the distal end 642 advancesthrough the working channel 639.

At block 920, the system 600 can calculate a predicted position changeor deflection of the distal end 632 of the leader 630 from the targetposition 618 based at least in part on the pose of the first instrument,or component(s) thereof. For example, the system 600 can calculate thepredicted position change based on the position and orientation of theleader 630 and/or sheath 620 (e.g., articulation angle 616 and/ortension in the pull wire 556). In another example, the system 600 cancalculate (or lookup in a database) the predicted position change basedon one or more of the physical properties of the system, such as thephysical properties of the leader 630 and/or insertable instrument 640(e.g., flexural rigidity of the insertable instrument 640 and/orflexural rigidity of the articulable region 638 of the leader 630),which in some examples may be coded into an RFID tag or the like on theleader 630 and/or insertable instrument 640 (read by an RFID reader orscanner, e.g., of the system 600).

At block 930, the system 600 can generate a control signal based on thepredicted deflection. The control signal can include instructions forreturning the distal end 632 back to the target position 618. In someembodiments, the control signal can include instructions for the drivemechanism 654 (or both drive mechanisms 654 and 664) to adjust atensioning of at least one of the pull wires 556 to compensate for thepredicted deflection and thereby return the distal end 632 back to thetarget position 618.

At block 940, the drive mechanism 500 of the system 600 can execute theinstructions in the control signal and adjust a tensioning of one ormore pull wires 556. In some embodiments, the instructions contained inthe control signal are executed in coordination with a determinableposition of the distal end 642 of the insertable instrument 640 withinthe working channel 639. For example, the control signal can be executedbefore the distal end 642 is advanced to the determinable position, thecontrol signal can be executed after the distal end 642 is advanced tothe determinable position, or the control signal can be executedconcurrently (e.g., incrementally) with the advancement of the distalend 642 through the working channel 639 of the leader 630.

Further Implementations

In accordance with one or more aspects, there is provided a roboticsystem that comprises: a first instrument that comprises (i) a shaftcomprising a proximal portion and a distal portion, the distal portioncomprising an articulable region, the shaft comprising a working channelextending therethrough, (ii) and at least one pull wire. The roboticsystem may further comprise at least one sensor configured to detect, inresponse to insertion of a second instrument into the working channel, aposition of a distal end of the second instrument within the workingchannel. The robotic system may further comprise at least onecomputer-readable memory having stored thereon executable instructions,and one or more processors in communication with the at least onecomputer-readable memory and configured to execute the instructions tocause the system to at least: calculate, based on a data signal from theat least one sensor, the position of the distal end of the secondinstrument within the working channel; and generate at least one controlsignal based on the calculated position. The robotic system may furthercomprise a drive mechanism connected to the at least one pull wire atthe proximal portion of the shaft, the drive mechanism configured to usea tensioning of the at least one pull wire based on the at least onecontrol signal, wherein the adjusted tensioning facilitates maintaininga position of the distal portion of the shaft.

In related aspects, the drive mechanism may be configured to use thetensioning of the at least one pull wire: as the distal end of thesecond instrument advances to a determinable position in relation to thearticulable region; before the distal end of the second instrumentadvances to the determinable position; and/or after the distal end ofthe second instrument advances to the determinable position.

In further related aspects, the one or more processors may be configuredto execute the instructions to cause the system to: detect an identifieron the second instrument; and generate the at least one control signalfurther based on the detected identifier.

In still related aspects, the one or more processors are configured toexecute the instructions to cause the system to determine at least onephysical property of the second instrument based on the detectedidentifier, wherein: the at least one physical property of the secondinstrument comprises a flexural rigidity value; and the one or moreprocessors are configured to execute the instructions to cause thesystem to generate the at least one control signal further based on theflexural rigidity value.

In yet further related aspects, the one or more processors areconfigured to execute the instructions to cause the system to: determinean articulation angle of the articulable region of the shaft; andgenerate the at least one control signal further based on thearticulation angle.

In additionally related aspects, the one or more processors areconfigured to execute the instructions to cause the system to detect theidentifier based on reading an RFID tag of the second instrument.

In related aspects, the robotic system may further comprise an EM fieldgenerator, wherein: the at least one sensor comprises a set of one ormore EM sensors at the distal end of the second instrument; and the oneor more processors are configured to execute the instructions to causethe system to calculate a position of the set of EM sensors within theEM field based on data from the set of EM sensors, and calculate theposition of the distal end of the second instrument within the workingchannel further based on the calculated position of the set of EMsensors.

In accordance with one or more aspects, there is provided a method ofcontrolling at least one pull wire of a first instrument, the methodcomprising: detecting insertion of a second instrument into a workingchannel of the first instrument, the second instrument comprisingproximal and distal ends, the first instrument, comprising a shaftcomprising proximal and distal portions, the distal portion comprisingan articulable region, and the at least one pull wire; calculating aposition of the distal end of the second instrument within thearticulable region; generating at least one control signal based on thecalculated position; and adjusting a tensioning of the at least one pullwire based on the at least one control signal, wherein the adjustedtensioning facilitates maintaining a position of the distal portion ofthe shaft.

In related aspects, the method may further comprise adjusting thetensioning of the at least one pull wire: as the distal end of thesecond instrument advances to a determinable position in relation to thearticulable region; before the distal end of the second instrumentadvances to the determinable position; and/or after the distal end ofthe second instrument advances to the determinable position.

In further related aspects, the method may further comprise: detectingan identifier on the second instrument; and generating the at least onecontrol signal further based on the detected identifier.

In yet further related aspects, the method may further comprisedetermining at least one physical property of the second instrumentbased on the detected identifier, wherein the at least one controlsignal is generated further based on the at least one physical property.The at least one physical property may comprise a flexural rigidityvalue of the second instrument. The detecting of the identifier maycomprise reading an RFID tag of the second instrument.

In still further related aspects, the calculated position of the distalend of the second instrument within the articulable region may be basedon data from at least one EM sensor on the distal end of the firstinstrument.

In accordance with one or more aspects, there is provided anon-transitory computer readable storage medium having stored thereoninstructions that, when executed, cause at least one computing device toat least, for a first instrument comprising at least one pull wire andan articulable region: detect insertion of a second instrument into aworking channel of the first instrument; calculate a position of adistal end of the second instrument within the articulable region;generate at least one control signal based on the calculated position;and adjust a tensioning of the at least one pull wire based on the atleast one control signal, wherein the adjusted tensioning facilitatesmaintaining a position of the distal portion of the first instrument.

In related aspects, the instructions that cause the at least onecomputing device to adjust the tensioning may cause the at least onecomputing device to adjust the tensioning of the at least one pull wireas the distal end of the second instrument advances to a determinableposition in relation to the articulable region.

In further related aspects, the instructions that cause the at least onecomputing device to adjust the tensioning may cause the at least onecomputing device to adjust the tensioning of the at least one pull wirebefore the distal end of the second instrument advances to thedeterminable position.

In yet further related aspects, the instructions that cause the at leastone computing device to adjust the tensioning may cause the at least onecomputing device to adjust the tensioning of the at least one pull wireafter the distal end of the second instrument advances to thedeterminable position.

In still further related aspects, the instructions that cause the atleast one computing device to adjust the tensioning may cause the atleast one computing device to: detect an identifier on the secondinstrument; and generate the at least one control signal further basedon the detected identifier.

In additionally related aspects, the instructions that cause the atleast one computing device to adjust the tensioning may cause the atleast one computing device to determine at least one physical propertyof the second instrument based on the detected identifier, wherein theat least one control signal is generated further based on the at leastone physical property. The at least one physical property may comprise aflexural rigidity value of the second instrument.

Implementing Systems and Terminology

Implementations disclosed herein provide systems, techniques andapparatus for improved navigation of lumens.

It should be noted that the terms “couple,” “coupling,” “coupled” orother variations of the word couple as used herein can indicate eitheran indirect connection or a direct connection. For example, if a firstcomponent is “coupled” to a second component, the first component can beeither indirectly connected to the second component via anothercomponent or directly connected to the second component.

The automatic compensation functions described herein can be stored asone or more instructions on a processor-readable or computer-readablemedium. The term “computer-readable medium” refers to any availablemedium that can be accessed by a computer or processor. By way ofexample, and not limitation, such a medium can comprise RAM(random-access memory), ROM (read-only memory), EEPROM (electricallyerasable programmable read-only memory), flash memory, CD-ROM (compactdisc read-only) or other optical disk storage, magnetic disk storage orother magnetic storage devices, or any other medium that can be used tostore desired program code in the form of instructions or datastructures and that can be accessed by a computer. It should be notedthat a computer-readable medium can be tangible and non-transitory. Asused herein, the term “code” can refer to software, instructions, codeor data that is/are executable by a computing device or processor.

The techniques disclosed herein comprise one or more steps or actionsfor achieving the described method. The method steps and/or actions canbe interchanged with one another without departing from the scope of theAlternatives. In other words, unless a specific order of steps oractions is required for proper operation of the method that is beingdescribed, the order and/or use of specific steps and/or actions can bemodified without departing from the scope of the Alternatives.

As used herein, the term “plurality” denotes two or more. For example, aplurality of components indicates two or more components. The term“determining” encompasses a wide variety of actions and, therefore,“determining” can include calculating, computing, processing, deriving,investigating, looking up (e.g., looking up in a table, a database oranother data structure), ascertaining and the like. Also, “determining”can include receiving (e.g., receiving information), accessing (e.g.,accessing data in a memory) and the like. Also, “determining” caninclude resolving, selecting, choosing, establishing and the like.

The phrase “based on” does not mean “based only on,” unless expresslyspecified otherwise. In other words, the phrase “based on” describesboth “based only on” and “based at least on.”

The previous description of the disclosed implementations is provided toenable any person skilled in the art to make or use the presentdisclosure. Various modifications to these implementations will bereadily apparent to those skilled in the art, and the generic principlesdefined herein can be applied to other implementations without departingfrom the scope of the disclosure. For example, it will be appreciatedthat one of ordinary skill in the art will be able to employ a numbercorresponding alternative and equivalent structural details, such asequivalent ways of fastening, mounting, coupling, or engaging toolcomponents, equivalent mechanisms for producing particular actuationmotions, and equivalent mechanisms for delivering electrical energy.Thus, the present disclosure is not intended to be limited to theimplementations shown herein but is to be accorded the widest scopeconsistent with the principles and novel features disclosed herein.

What is claimed is:
 1. A robotic system, comprising: a first instrumentcomprising: a shaft comprising a proximal portion and a distal portion,the distal portion comprising an articulable region and a distal end,the shaft further comprising a working channel extending through alength thereof; and at least one pull wire; at least one sensor; atleast one computer-readable memory having stored thereon executableinstructions; one or more processors in communication with the at leastone computer-readable memory and configured to execute the instructionsto cause the system to at least: determine, based at least in part onone or more data signals from the at least one sensor, a position of atleast a portion of the first instrument relative to at least a portionof a second instrument disposed within the working channel of the firstinstrument; and generate at least one control signal based at least inpart on the determined position; and a drive mechanism connected to theat least one pull wire and configured to adjust a tensioning of the atleast one pull wire based at least in part on the at least one controlsignal.
 2. The robotic system of claim 1, wherein said adjusting thetensioning of the at least one pull wire facilitates returning thedistal end of the distal portion of the shaft towards an initialposition before a position change of the distal end.
 3. The roboticsystem of claim 2, wherein at least one control signal comprisescommands for the drive mechanism to increase the tensioning of the atleast one pull wire until the distal end of the distal portion of theshaft is returned to the initial position as measured by the one or moredata signals from the at least one sensor.
 4. The robotic system ofclaim 1, wherein the one or more processors are further configured toexecute the instructions to cause the system to detect a first positionchange of the distal end of the distal portion of the shaft in responseto insertion of the second instrument into the working channel of theshaft.
 5. The robotic system of claim 4, wherein: the second instrumentfurther comprises one or more electromagnetic (EM) sensors; and the oneor more processors are further configured to execute the instructions tocause the system to: calculate a second position of the one or more EMsensors within an EM field based on data from the one or more EMsensors; and generate the at least one control signal further based atleast in part on the calculated second position.
 6. The robotic systemof claim 4, further comprising at least one respiration sensor, whereinthe one or more processors are further configured to execute theinstructions to cause the system to: determine, based on data from theat least one respiration sensor, a respiration pattern of a patientduring acquisition of at least one of the one or more data signals fromthe at least one sensor; and distinguish the first position change ofthe distal end of the distal portion of the shaft in response to theinsertion of the second instrument into the working channel from asecond position change of the distal end of the distal portion of theshaft caused by the respiration pattern of the patient based at least inpart on the respiration pattern.
 7. The robotic system of claim 4,wherein the one or more processors are further configured to execute theinstructions to cause the system to: detect an identifier on the secondinstrument; and generate the at least one control signal further basedat least in part on the detected identifier.
 8. The robotic system ofclaim 1, wherein: the drive mechanism is connected to an end effector ofa robotic arm; and the robotic arm and the drive mechanism areconfigured to navigate the distal portion of the shaft through a luminalnetwork of a patient to a treatment site.
 9. The robotic system of claim1, further comprising an electromagnetic (EM) field generator, wherein:the at least one sensor comprises a first set of one or more EM sensorsat the distal end of the shaft; and the one or more processors arefurther configured to execute the instructions to cause the system to:calculate a first position of the first set of EM sensors within an EMfield based on data from the first set of EM sensors; and detect aposition change of the distal end of the shaft based on the calculatedfirst position.
 10. The robotic system of claim 1, wherein: the at leastone sensor comprises a set of one or more inertial sensors at the distalend of the shaft; and the one or more processors are further configuredto execute the instructions to cause the system to: calculate a firstposition of the set of one or more inertial sensors based on data fromthe set of one or more inertial sensors; and generate the at least onecontrol signal further based at least in part on the calculated firstposition.
 11. The robotic system of claim 1, wherein: the at least onesensor comprises a set of one or more strain gauges; and the one or moreprocessors are further configured to execute the instructions to causethe system to: calculate a first position of the distal end of the shaftbased on data from the set of one or more strain gauges; and generatethe at least one control signal further based at least in part on thecalculated first position.
 12. The robotic system of claim 1, wherein:the first instrument comprises an endoscope; and the at least one sensorcomprises a set of one or more cameras at the distal end of theendoscope.